1. Field of the Invention
The present invention relates to a composition technique of a physical space image and virtual space image.
2. Description of the Related Art
In recent years, as a technique for seamlessly merging a physical world and virtual world in real time, a so-called MR (Mixed Reality) technique is known. As systems that implement the MR technique, two systems have been mainly proposed. One is an MR system using an optical see-through HMD (Head Mounted Display), and the other is an MR system using a video see-through HMD. The system using the optical see-through HMD presents, to the HMD user, an MR image generated by superimposing a CG (Computer Graphics) image onto an image of the physical space observed by the HMD user and formed on the pupils of the user via an optical system. The system using the video see-through HMD presents, to the HMD user, an MR image generated by superimposing a CG image onto an image obtained by sensing an object that nearly matches an object observed from the pupil position of the HMD user using a video camera or the like.
Either of these MR systems detects the position and orientation of the viewpoint of the HMD user by making arithmetic processing of an output signal from a sensor and the like mounted on the HMD using a PC or the like. Each system generates a CG image to be superimposed at a predetermined position of the physical space observed by the HMD user based on the detection result in the PG, and composites the generated CG image on a physical image using, for example, a chroma-key composition method. In this manner, the aforementioned MR system enhances the sense of reality. However, the detection of the position and orientation of the viewpoint of the HMD user, and the generation of a CG image require a lot of time. Therefore, when a CG image is to be composited on a physical space image which requires little processing time, a dynamic registration error occurs between these images. Especially, when the orientation (direction) of the viewpoint of the HMD user has changed, an error associated with a relative position between the physical space image and CG image due to the dynamic registration error occurs, thus impairing the sense of reality.
To solve such a problem, patent reference 1 has proposed the following method. That is, the head azimuth direction of the HMD user is detected, and a CG image is shifted in the horizontal and vertical directions by the moving amount of the head of the HMD user within a time required to generate the CG image based on the detection result, and is then composited on a physical space image.    [Patent Reference 1] Japanese Patent Laid-Open No. 2004-109994
However, the aforementioned related art suffers the following problem.
When a CG image to be composited on a physical space image is a three-dimensional object, an inconsistency occurs between the stereoscopic angle that represents the depth of the CG image and the composition position on the physical space image if the CG image is merely shifted in the horizontal and vertical directions by the moving amount of the head of the HMD user within the time required to generate the CG image. For this reason, the sense of reality is impaired.